Outline of behaviour sensitivity analysis

System dynamics models are described as being “causally-descriptive”, meaning that unlike other “black-box”, it is not sufficient that system dynamics only produce accurate output. In addition to being numerically valid, system dynamics models’ behaviour needs to be validated.

The test presented in this document examines which input assumptions have the greatest impact on model behaviour, unlike previous sensitivity analysis, which focuses on numerical relationships between input assumptions and outputs (either model outcomes or policy outcomes).

Outcome Measure

The behaviour of each model test was evaluated using the cumulative difference of the first derivative. The following plots explain how this outcome measure is calculated.

Figure panel A shows a plot of three outcomes, the base model, with no changes to any initial assumptions. The result varies an input assumption that produces similar model behaviour and a second test, where varying the input assumption produces different model behaviour. The absolute cumulative difference between the model outcomes is plotted in panel C.

Panel B plots the first derivative of each model result. The model results with similar behaviour also have a similar first derivative. Conversely, the different behaviour result has differences in the derivative. The cumulative absolute difference of the first derivative shows a greater difference between the two model results highlighting the difference between behaviours.

While the cumulative absolute difference measure cannot give insights into the model structure, it is useful to screen input assumptions that impact model behaviour .

Unlike the extreme test, this analysis focuses on determining which input assumption impacts the model behaviour.

Example plot for model behaviour

How to interpret the results

Each input assumption is sequentially sampled 20 times between +/- 20% of the initial assumed input value. The cumulative difference of the first derivative is used to assess how each variable sample impacts model behaviour and the average cumulative difference between the 20 samples are summarised in the result plots.

The figure below steps through how the summary plots are calculated. The summary plot (subplot A) shows the average cumulative difference between the 20 samples of the metabolic equivalence of task (MET) for vigorous physical activity. This summary plot is presented for all tested variables.

The outcome, prevalence of obesity (subplot B), is plotted for each tested assumption of vigorous MET value across each age group. The subsequent derivative (subplot C) and cumulative difference in the derivative are also plotted (subplot D), with the average incremental difference plotted in red. This is the summary value presented in the summary plots.

A high average cumulative difference in the first derivative suggests a variable with a larger impact on model behaviour.

Overall Ranking of sensitivity

The 649 input assumptions tested can be summarised into 70 variables by taking the mean of cumulative difference in derivatives age-gender-BMI specific inputs, resulting in 70 overarching variables. These are ranked in the following table.

Below is the ranking of aggregate input assumptions, where input assumptions with multiple input subgroups have been averaged. For example, changes in body weight are tested by age-gender-BMI subgroups however, are ranked based on the mean cumulative difference in first derivatives.

The results show that variables that have a global impact, i.e. those that impact all of the modelled population, have the greatest impact on model behaviour.

Examining the most sensitive variable: energy density of sugar

Body Weight

Change in Male Body Weight input assumptions

Change in Female Body Weight input assumptions

Height

Change in Male height input assumptions

Change in Female height input assumptions

Growth Function kJ/day

Macronutrient energy density

Thermic effect of food (TEF)

Adult to Adult Social Transmission

Adult to Child Social Transmission

Child to Child Social Transmission

Infant Reported behaviours

Assumed intergenerational relationships

Mortality ratios

The Metabolic Equivalent of Task (MET)

Light physical activity

Moderate physical activity

Vigorous physical activity

Screen time

Sleep

Fruit Reported Intake

Vegetables Reported Intake

Grains Reported Intake

Dairy Reported Intake

Meat and Protein Reported

Discretionary foods Reported

Fats and Oils Reported

Sugar based beverage Reported

Water Reported

Initial fat-mass (%)

Change in Male fat-mass % assumptions

Change in Female fat-mass % assumptions

Proportion of nutrients within food group Inputs

Grains

Vegetables

Fruit

Dairy

Meat and Protein

Fats

Discretionary foods

Sugar-sweetened beverages

Miscellaneous (Other)

Non-sugar-sweetened beverages

Initial BMI Prevalence Inputs

Change in Male Initial BMI Prevalence

Change in Female Initial BMI Prevalence